1. Field of the Invention
The present invention relates generally to pattern classification systems and, more particularly, to a compound pattern classification system using neural networks which is able to vary a response signal by learning from repeatedly input pattern signals to provide correct classification results.
2. Description of the Prior Art
Pattern classification systems, such as character or voice recognition systems, separate and identify classes of incoming pattern signals. FIG. 1 shows a conventional pattern classification system such as described by Richard O. Duda and Peter E. Hart in Pattern Classification and Scene Analysis, Wiley-Interscience Publishers, pp. 2-4. This classification system includes an input transducer 1, such as a television camera, which performs opto-electronic conversion of characters to generate pattern signals S providing characteristic information about the characters. The system further includes a feature extractor 2 which receives the pattern signals S and generates feature vectors F useful for classifying the characters. The system is also provided with a classifier 3 which classifies the characters and generates classification responses P based on the distributions of the feature vectors F. In order to make such classifiers, pattern recognition techniques, such as a linear discrimination method, have been developed. However, the classification systems using these techniques are unable to learn by adjusting classes to account for new input patterns or to create new classes. Consequently, it is necessary to manually develop the information for classifying pattern signals and manually incorporate the information into the system. This manual development and incorporation diminishes the efficiency of the system and provides another potential source for error (i.e. human error).
In order to solve this problem, many self-organizing pattern classifiers have been proposed which are able to organize themselves correctly to separate a given number of pattern signals into their classes. An example of a self-organizing pattern classifier is that which make use of a back propagation learning method such as shown by Richard P. Lippmann in "An Introduction to Computing with Neural Nets," IEEE ASSP Magazine, April 1987, Vol. 4, No. 2, pp. 4-22. The back propagation technique is an iterative gradient algorithum that seeks to minimize the mean square error between actual output and desired output. Another example of a self-organizing pattern classifier is the learning vector quantization 2 technique such as shown by Teuvo Kohonen, Gyorgy Barna, and Ronald Chrisley, "Statistical Pattern Recognition with Neural Networks: Benchmarking Studies," Proceedings of IEEE International Conference on Neural Networks, Jul. 24-27 1988, Vol. 1, pp. 61-68.
These self-organizing pattern classifiers suffer the drawback that when they make a wrong classification, they modify the information about stored weighting data to attempt to yield more accurate results. FIG. 2 shows the distributions of two classes C.sub.A and C.sub.B in a two dimensional vector space defined by feature axes X1 and X2. The above self-organizing classifiers are able to make correct boundaries 9 by using the back propagation learning method or learning vector guantization 2 technique (referenced above) to separate the two classes C.sub.A and C.sub.B.
As long as the distributions of the respective feature vectors F, each consisting of N elements f1, f2, . . . , fN, do not overlap each other, the above classifiers are able to learn to provide correct classification with high classification rates. However, as FIG. 3 shows, when the distributions 10 and 11 of the feature vectors F of two classes C.sub.A and C.sub.B overlap each other in an area 12, none of the above learning techniques make it possible to separate these two classes.
When a large number of classes are identified, such as are used with classifying Chinese characters, it is rare for a feature vector of a given class to not overlap with feature vectors of other classes (hereinafter such feature vector will be referred to as a "single aspect feature vector"). Thus, the above described self-organizing classifiers which have been designed for single aspect feature vectors fail to provide high recognition rates for multiple aspect feature vectors.
One approach to overcoming this problem of overlapping feature vectors of different classes is to utilize multiple features. FIG. 4 provides an example wherein a single feature is used. In particular, it shows the distributions 13 and 14 of brightness features F1 for ash wood and birch wood respectively, as described in Pattern Classification and Scene Analysis, at pp. 2-4. FIG. 5 provides an example wherein multiple features are used. FIG. 5 shows the distributions 15 and 16 of ash wood and birch wood, respectively, with respect to the brightness feature F1 and the grain prominence feature F2. In FIG. 4, there is a large overlapping area in the brightness feature F1 of the ash wood 13 and the birch wood 14. As such, it is impossible to make correct classification using only the brightness feature F1. However, as shown in FIG. 5, by using both the brightness feature F1 and the grain prominence feature F2, it is possible to classify these two objects correctly. In this way, by inputting two or more feature vectors, the use of multiple features does not suffer the drawback described above for single feature approaches.
However, a drawback with the use of multiple feature vectors is that the features of feature vectors F1, F2, . . . , FN (where N is a positive integer) are not generally related. As a result, not only large areas of memory but also large amounts of computing time are necessary to input the N feature vectors F1, F2, . . . , FN into the self organizing pattern classifier. For example, suppose that there are no relations among the three feature vectors F1, F2, and F3 which are used to identify an object A. Further suppose that the object A has four different instances F11, F12, F13, and F14 of the feature vector F1, four different instances F21, F22, F23, and F24 at the feature vector F2, and four different instances F31, F32, F33, and F34 of the feature vector F3. Then, it is possible to represent the object A by using a vector F as follows: EQU F={F1i, F2j, F3k)}
wherein i, j, k=1, 2, 3, 4. Thus, there are 64 (i.e. 4.times.4.times.4) different instances, and 192 (i.e. 64.times.3) vectors are required to represent the object A.
Accordingly, it is an object of the invention to provide a compound self-organizing pattern classification system that effectively utilizes a plurality of different feature vectors.
It is another object of the invention to provide a compound self-organizing pattern classification system which is able to classify pattern signals with accuracies higher than those of respective pattern classifiers by making compound classification based on the outputs of a number of independent self-organizing pattern classifiers into which a number of different feature vectors are input.